vfs: add ZipProvider - #64915

Codecov Report

❌ Patch coverage is 97.93814% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.11%. Comparing base (a844473) to head (2dc89ec).
⚠️ Report is 10 commits behind head on main.

@@            Coverage Diff             @@
##             main   #64915      +/-   ##
==========================================
+ Coverage   90.07%   90.11%   +0.04%     
==========================================
  Files         754      755       +1     
  Lines      256378   257134     +756     
  Branches    48498    48784     +286     
==========================================
+ Hits       230936   231720     +784     
+ Misses      16572    16564       -8     
+ Partials     8870     8850      -20     

Note: the failing mac-os test seems to be a flakiness issue with that test. I've seen it before and it seems to be entirely unrelated.

@mcollina given you are the VFS man, this one is for you.

Note:

Next up will be a commandline option --vfs-mount which allows mounting a VFS mount from the CLI for the app to use. This is useful for example to provide an app with an archive of files it can use, like bundled resources. This can be mounted by either mounting a directory (development) or a bundled archive (deployments)

Following on from that is a --vfs-load flag that will set the entrypoint to a vfs mounted volume. In the full combination it's possible to have bundled node apps (zip archives) that get loaded. --vfs-mount already allows registering custom vfs-providers (via --import or -r). So what the vfs-provider does (signed code checking, etc...) is up to the user. In the simplest case, you just have an app as a zip archive and it works out of the box.

I have all the code already and it's working quite nicely.

I'm doing significant surgery on the vfs machinery in #63653.

I need to land that before this.

I'm doing significant surgery on the vfs machinery in #63653.

I need to land that before this.

Have a look at my branch pipobscure#8 here which does all the module loader work for VFS already. And I've been testing it like crazy.

Though from what I see in your PR you're going way deeper, especially the fact that you have an O(1) identification of vfs based modules is a much better solution. So I'm EXTREMELY happy to hold on with this and then rework my other PRs based on your stuff 😄 (starting work based of your solution)

Note: It looks like #63653 is entirely orthogonal to this PR as that doesn't alter the Provider surface contract at all. But for sanity sake it's still better to hold this until #63653 is landed and rebase this on that then.

CI: https://ci.nodejs.org/job/node-test-pull-request/75379/

CI: https://ci.nodejs.org/job/node-test-pull-request/76771/

Rebased on main because CI was failing with an unrelated test.

I am excited to use this in combination with the import bytes proposal: you can have an on-disk .zip that you import with import-bytes and load as a VFS, and that will run as-is, and you can also trivially bundle it into a single .js file (with something like esbuild) for distributing.

I am excited to use this in combination with the import bytes proposal: you can have an on-disk .zip that you import with import-bytes and load as a VFS, and that will run as-is, and you can also trivially bundle it into a single .js file (with something like esbuild) for distributing.

Have a look at https://github.com/pipobscure/bundles and https://claude.ai/code/artifact/7860e0a8-3d32-4b9e-b5ce-94a36289bc34?via=auto_preview to get a preview of why I'm doing this stuff and what the plan is 😄

And if you want a node that can do exactly this build https://github.com/pipobscure/node/tree/deploy

I apologize for the force push. I was experimenting with ZipProvider in my local workspace, and pushed to pipobscure's fork instead of my fork. It's back to previous state.

This needs a re-review as rebase was done 12 hours ago as per #64915 (comment)

CI: https://ci.nodejs.org/job/node-test-pull-request/76800/

I have no way to view CI, so I can't investgate/validate what is happening with the 3 failures. Given that they fail equally on #64915 and #65680 I am assuming that this might be a pure CI issue and nothing in the PRs. However that's a random guess. Can someone please confirm that intuition? I am happy to fix anything I can.